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In 1983 the Maryland State Department of Public 
Education (MSDE) issued a request for proposals for "The Development 
of the Score Reporting System for the Maryland Functional Testing 
Program." The MSDE called for a literature review, a nation >1 survey, 
a statewide survey of user needs and capabilities, an assessment of 
the state's report producing capability, and a final design for 
reports and a user's manual. Following a literature search, national 
and statewide surveys of reporting practices and information needs 
were conducted by Measurement Incorporated. Common and unique needs 
of district and building cdministrators, teachers and counselors, and 
parents and students were found. Using the nationwide search results, 
the information needs of students, parents, teachers, guidance 
counselors, principals, and district administrators in Maryland were 
surveyed. Score report design was based upon these studies 
emphasizing the accountability function of the tests. Four levels of 
reporting and seven content areas necessitated 28 separate score 
reports. Examples of four levels of reports (student, class, school, 
and local education agency) are presented. Each report is oriented to 
a specific audience, visual clutter is reduced, and diagnostic 
information is briefly presented. A usee's guide provides thorough 
background on score interpretation at multiple levels. This score 
reporting system appears to meet the responsibilities and information 
needs of all its audiences. (PN) 
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ABSTRACT 



While most states and many local school districts have criterion 
referenced testing programs, little is known about how or why scores are 
reported. In revising one staters reporting procedures, we conducted national 
and statewide st rveys of reporting practices and information needs. We found 
common and unique needs of district and building administrators, teachers and 
counselors, and parents and students. Using the results of the nationwide 
search, we surveyed the information needs of students, parents, teachers, 
counselors, principals, and district administrators in one state. Score report 
design was bas^^d upon both the national and statewide studies. User groups 
found the reiiulting 28 score reports to be helpful. 

While previous studies had suggested that information provided by many 
score reporting systems is deficient in both quantity and quality, this study 
suggests that users are receiving as much information as they can absorb but 
that it is often not the right information. The score reports designed as a 
result of this study are quite brief and reflect the accountability function of 
the testing program. Procedurts for interpreting^ results and obtaining 
additional diagnostic information are described. 
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Introduction 

In 1983 the Maryland State Department of Public Education (MSDE) issued a 
request for proposals (RFP) for "The Development of The Score Reporting System 
for the Maryland Functional Testing Program." The RFP called for a literature 
review, a national survey, a statewide survey of user needs and capabilities, 
an assessment of the staters report producing capabil:</-y, and a final design 
for reports and a user's manual. A contract to implement this project was 
ultimately awarded to Measurement Incorporated and its subcontractor RMC 
Research Corporation. 

The results of the literature search and nationwide survey of 
criterion-referenced test score reporting were reported two years ago (Haenn, 
Bunch, and Mengel, 1984). Those results are briefly summarized here. While 
the literature on test score reporting is scant, some generalizations may be 
gleaned: 1) every score report should describe the testing program, 2) 
presentation of results should be audience £:pecific, 3) each score report 
should contain cautions to observe in interpretation. 

With regard to the first point, it is important that each score report 
recipient know why he or she is receiving the report. Reports should contain 
the purpose of the test, the name of the test (specific level, form, edition, 
etc.), standardization date, and administration date. Reports should also 
explain how and why results are to be used in language the recipient (audience) 
can comprehend. 

Five levels of audience are identified: student (or parent), class, 
school, district, and state. At each level the psychometric and content 
sophistication levels are different. What is meaningful or relevant to one 
audience is not necessarily so to another. Score reports must be designed with 
this fact in mind. 
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Finally, since all school personnel and most parents have preconceived 
no».ions about meanings of test scores, there is a real danger that results will 
be misinterpreted. Producers of score reports need to explain their score 
netrics; chis includes narrowing the range of permissible Interpretations. 
Caveats and precise language are particularly important because most large 
scale testing programs are routinely reported by the news media. 

With these three points in mind, we developed a set of questionnaires to 
assess the state of the art of score reporting in the fifty states. We paid 
careful attention to what Mills and Hambleton (1980) had considered important 
information needs for our five audiences. The end products were two 
questionnaires, one for state directors of testing and one for local directors. 

We received responses from 28 out of ^9 states (Maryland was excluded) and 
27 out of 57 large districts. After a series of telephone follow-up 
interviews, only five states with statewide criterion-referenced testing 
programs were unaccounted for. Specific results were reported by Haenn, 
Bunch, and Mengel (1984) . 

We found that the typical score reporting system yielded only 20-30% of 
the items listed by Mills and Hambleton (1980). Some respondents who had no 
system reported on what they would like to see. Even these respondents 
endorsed only 40-50% of the items on the questionnaires. 

Several states and districts supplied sample score reports or supporting 
materials. These included manuals, labels, guides, and one bilingual report 
form. We identified a number of desirable and possibly useful features for 
subsequent review by Maryland audiences. 



Statewide Survey 



In conducting the statewide survey, we wanted to answer three questions: 
1) Who are the score report users? 2) What do they need to know? 3) How much do 
they understand about testing In general and about the Maryland Functional 
Testing Program In particular? To answer these questions > we surveyed over 
1,000 Individuals representing four of the five reporting levels mentioned 
earlier. At the same time we met with MSDE staff to determine *^ -^chnical and 
personnel capacity to produce a variety of score reports. 



Sample 

Maryland has 24 local education agencies (LEAs) and just over 500 
secondary schools. Our survey Included all 24 districts and 105 secOi\dary 
schools. Table 1 shows the size and nature of the total sample. 



Table 1 
Sample Size by Audience 



Audience Sample 

Superintendents 24 

Assistant Superintendents 20 

Project Basic Facilitators 24 

Local Accountability Coordinators 24 

Other Administrators 40 

Principals 105 

Guidance Counselors 105 

Teachers 400 

Students 200 

Parents 200 

TOTAL 1,142 



A twenty percent random sample of all non-elementary schools yielded 105 
schools, thus, 105 principals and guidance counselors. The number of 
superintendents and other district level administrators shown In Table 1 
represents nearly a 100% sample of superintendents, assistant superintendents 
of Instruction, Project Basic facilitators, local accountability coordinators. 



and other administrators* Each administrative unit (23 counties and Baltimore 
City) was represented in each sample in proportion to its non- elementary school 
enrollment. 

Questionnaires 

Separate questionnaires were developed for district administrators, 
principals, counselors, teachers, parents and students. District 
Administrators received a one-page form requesting information about what is 
reported and what should be reported. Principals were asked to tc:il what kinds 
of information they needed. Specific items of information were listed and 
principals were asked to check those which they considered necessary. 

Teachers and counselors were asked to rank items in terms of importance. 
They were also asked what they would be willing to do to obtain more or better 
information, given the fact that one or two hours of testing cannot yield 
detailed assessments of every instructional objective. 

Parents and students received a one-page questionnaire with ten items. 
Their task was to select the five items they thought were t^-^ most important 
(e.g., statement telling you whether you pacsed or failed; topics you need to 
study) • 

Procedure 

Upon approval by the MSDE, we mailed packets of materials to the 24 local 
superintendents of schools* Each packet contained the superintendent's 
questionnaire, a cover letter from State Superintendent David Hombeck, a 
sample set of all additional questionnaires, and one or more school 
'questionnaire packets* Each school packet was to be sent to principals by 
their customary methods Questionnaires to assistant superintendents and other 
local administraf -^rs were mailed separately. 



Each school packet contained the principal's questionnaire, a cover 
letter, a sample set of parent, student, teacher, and guidance counselor 
questionnaires, and appropriate numbers of other questionnaiies and related 
materials. Each questionnaire was accompanied by a plain white envelope with 
the type of questionnaire printed on the front. All correspondencs were asked 
to return their questionnaires in the attached envelope to the school office. 
Principals were asked to place all returned envelopes in ^ larger stamped 
envelope addressed to KMC Research Corporation. Thus, effort by respondents 
was minimiz'^d. 

Data Analysts 

The primary form of analysis for the data obtained in this survey was 
frequency distribution. No attempt was made to cross tabulate within or across 
forms of audience or to correlate responses of one group with those of another. 
The reason for this approach is t^at many parts of the questionnaires required 
choices among viable features, s rhemrre, it is unlikely that any report can 
or should do all things for a. .le. Therefore, soma choices regarding 

features must be made. Only the mc X desirable or necessary features should be 
included in reports if a primary feature is simplicity or clarity. 

Results 

Response rate . Overall response rate was very good (approximately 54%) 
given the rather short time allowed and the totally voluntary nature of the 
survey. Table 2 summarizes response rate by audienca. 
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Table 2 
Response Rate by Audience 



Percent of 





Questionnaires 


Quest lonna j res 


Questionnaires 


Audience 


Sent 


Returned 


Kecurnea 


Superintendents 


24 


10 


A 0 


A88l8f:ant Superintendents 


on 


7 


JJ 


Local Accountability 








Coordinators 


24 


7 


29 


Project Basic Facilitators 


24 


8 


33 


Other Administrators 


40 


24 


33 


Principals 


105 


64 


60 


Guidance Counselors 


105 


61 


58 


Teachers 


400 


219 


55 


Parents 


200 


99 


55 


Students 


200 


119 


60 


Total 


1,142 


618 


54 



As can be seen from Table 2, response rate was higher at the school level 
and lower at the district administrative levels. Furthermore, responses at the 
school level were fairly evenly spread throughout the state, while 
administrative responses were not. For these reasons, results pertaining to 
principals, guidance counselors, teachers, parents, and students can be 
confidently generalized to the state as a whole. Responses of district 
administrators are not directly generallzable unless considered all together. 
Even then they are somewhat Idiosyncratic. With this caution In mind, we now 
turn to the tabulated results. 

District administrators . Table 3 summarizes the responses of all 
district level administrators. These five groups (superintendents, assistant 
superintendents. Project Basic facilitators, local accountability coordinators 
and other administrators) have been combined because the response rate of each 
group was very low. The other administrators group was composed primarily of 
curriculum and content area supervisors. By combining all administrative 
groups, we hoped to stabilize the results. 
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Table 3 

Responses of District Administrators 
(N « 56; entries are percentages) 



What information would be helpful in analyzing your district's performance 
on the Maryland Functional Tests? Please check all boxes that apply. 





IT V L ca^ii 




For the 

Qt*^ t*P 


Averc'.ge xouaJ. ocoxe 




Q1 

7 J. 


o u 


iWcLdjgc iyUuiclJ.Il Ol»LfLc 




7 J 


70 


AvpTflOP Oblpptfvp Spotp 


88 


88 


68 


Item Scores 


70 


6A 


A3 


Strengths /Weaknesses 








Domains 


80 


71 


5A 


Objectives 


73 


71 


Al 


Items 


55 


5A 


27 


Pass/Fail 








Total Test 


93 


96 


80 


Domains 


93 


88 


6A 


Objectives 


8A 


79 


52 


Past Performance 








Average Score 


95 


93 


82 


Number or Percent Passing 


100 


93 


75 


Number or Percent Failing 


91 


89 


68 



From your perspective, which of the following items of information about 
the Maryland Functional Tests should be included on the score report? 



Yes No Yes 

1. Why the test was given 
2* What the passing score was 

3» Who will know about my district's performance 
4* Resources and support available for those 

who perform poorly 
5. Resources and support available for 

interpretation of the scores 



66 


30 


3 


98 


0 


1 


58 


23 


17 


66 


23 


10 


82 


12 


5 



7 

10 



Three jints are Ijnmediately clear. First there was greater interest in 
local performance than in state performance. Second, there was major interest 
in information about past performance. Third, interest in performance was 
greatest at the highest level of generality (i.e., total score, pass/fail), 
slightly less at intermediate levels (i.e., domain), and least at the lowest 
level of generality (i.e., objectives, items). 

With respect to Part B of Table 3, district administrators were primarily 
interested in knowing the passing score ror each test and relatively less 
interested in other matters. There was great interest in remediation 
resources, however. 

Principals . Responses of principals are summarized in Table A. As with 
district administrators, the most appropriate way to interpret principals' 
responses is in relative terms. 

Table 4 
Responses of Principals 
(N » 64; entries are percentages) 

A. What performance would be helpful in analyzing your scI»ool's performance 



on the Maryland Functional Tes 


ts? Please 


check all boxes 


that apply. 




For your 


For your 


For the 




school 


district 


state 


Average Total Score 


84 


•/s 


70 


Average Domain Score 


81 


65 


56 


Average Objective Score 


81 


65 


54 


Item Scores 


82 


56 


46 


Strengths/Weaknesses 


Domains 


85 


62 


50 


Objectives 


87 


62 


48 


Items 


85 


53 


40 


Pass/Fail 


Total Test 


93 


71 


64 


Domains 


78 


53 


42 


Objectives 


73 


46 


40 


Pest Performance 


Average Score 


90 


73 


62 


Number or Percentage Passing 


93 


71 


57 


Number or Percentage Failing 


87 


64 


50 
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Table 4 Continued 

B. From your perspective, which of the following Items of Information about 
the Maryland Functional Tests should be Included on the score report? 







Yes 


No 


Omit 


1. 


Why the test was given 


67 


25 


7 


2. 


What the passing score was 


98 


0 


1 


3. 


Who will kpow about my school's performance 


70 


21 


7 


4. 


Resources and support avallabls for those who 










perform poorly 


85 


9 


4 


5. 


Resources and support available for intepreta- 










tion of the scores 


90 


6 


3 
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There was a general progression In Interest which was highest at the 
source (school), lower at the district level, and lowest at the state level. 
This phenomenon Is understandable given the fact that principals have the 
greatest potential Impact on future result; at their own scnools. Other trends 
parallelled those observed with district administrators; namely, greater 
Interest In past performance, major Interest In total test scores and domain 
scores relative to objectives and Items, and Intense Interest In passing scores 
and available resources Items (B2 and B5) . 

Guidance counselors Responses of guidance counselors are summarized In 
Table 5* The method of Interpretation used with other audiences Is appropriate 
here as well* 

Table 5 

Responses of Guidance Counselors 
(N=61; entries are percentages) 

l.A* Listed below are six Items that could appear on Individual student test 
reports. Consider each and circle THREE (3) that you think are the most 
Important FOR INDIVIDUAL STUDENT TEST REPORTS. 

If you circle more than three > we cannot r unt your rerponses* You may 
circle fewer than three if you wish. 

87 1, Total score (e.g., reading, mathematics) 
37 2. Domain scores (e.g., number concepts, decimal operations, 
using data 

20 3, Objective scores (e.g., using information from tables, using 

information from graphs) 
05 4. Item scores 
58 5. Strengths and weaknesses 
65 6. Pass/Fail Indicator 

9 12 



Table 5 Continued 



B. Consider the types of SUMMARY INFORMATION shown below. Which would be 
useful to you? Check all boxes that apply. 



For your For your For the 

school district state 

Average Total Score 82 68 §2 

Average Domain Score 73 55 47 

Average Objective Score 52 38 28 

Item Scores 52 35 25 



Strengths/Weaknesses 

Domains 77 40 33 

Objectives 57 [ 33 JO 

iZems 55 30 23 



Pass/Fail 

Total Test 90 68 67 

Domains 62 47 42 

Objectives 45 38 ^6 

Past Performance 

Average Scores 75 53 55 

Number or X Passing 82 57 53 

Number of % Failing 77 55 48 



II. Which of the following '-ays ol reporting student strengths and weaknesses 
is most helpful to you? (CIRCLE ONE ONL£) 

20 A. Relative to the student's total score (e.g., one domain 
score is lower than you would have expected, given the 
student's total score) 

68 B, Relative to the passing score (e.g., one or more domain scores 
are below a certain standard) 

13 C. Relative to other studentb* scores (e.g., this student scored 
higher than the average student on one domain but lower on 
cnother) 

III. Which of the following would you be willing to do in order to get more 
information about students* strengths or weaknesses at the objective 
level? (CIRCLE ALL THAT APPLY) 

13 A. Give a longer test 

50 B. Give a test that covers fewer objectives but covers each more 
completely 

57 C. Give follow-up tests for low-scoring stvdents 
08 D. Other 
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The primary interests of guidance counselors were total score and 
pass/i'ail information (Part I. A >. Domain sjores and information about 
strengths and weaknesses were secondary concerns. There was very little 
interest in objective scores (20%) or item information (5Z) . 

As with principals, counselors were primarily interested in the students 
in their own schools and les3 so in other students in the district or state 
(Part I*B*)* Major emphasis was on pass/fail information, though interest in 
total scores, past performance (Z passing), and domain scores was relatively 



The two quesf'ons on page 2 of the questionnaire yielded very helpful 
information. The vast majority of counselors (68%) preferred to view strengths 
and weaknesses in terms of some absolut ^ standard (response B) rather thv ^ in 
normative terms (response C - 13%) . In order to receive more detailed 
information about student performance on specific objectives, counselors were 
fairly evenly divided between a more focused test (response B - 50%) and 
follow-up tests for selected students (response C - 57%). Few would have given 
a lotiger test (re ^onse A - 13%). 

T o.achers . Respons es of teachers are summarized in Table 6. 



high. 



U 




Table 6 
Responses of Teachers 
(N«219; entries are percentages) 

Listed below are six Items that could appear on Individual student test 
reports. Consider each and circle THREE (3) that you think are the most 
important FOR INDIVIDUAL STUDENT TEST REPORTS* 

I. A. It you circle more than three, we cannot count your responses. You may 
circle fewer than three If you wish. 



65 


1. 


Total score (e.g., reading, mathematics) 


53 


2. 


Domain scores (e.g., n>-jnber concepts, decimal operations. 






using data, problem solving) 


?8 


3. 


Objective scores (e.g., using information from tables, using 






information from graphs) 


18 


4. 


Item scores 


56 


5. 


Strengths and weaknesses 


38 


6. 


Pass/Fail indicator 



B. Consider the types of SUMMARY INFORMATION shown below. Which would be 
useful to you? Check all boxes that apply. 

For Your For Your 

School District Total State 



Average Total Score 


76 


68 


66 


Average Domain Score 


70 


48 


40 


Average Objective Score 


63 


41 


30 


Item Scores 


62 


30 


26 


Strengths/Weaknesses 








Domains 


75 


49 


41 


Objectives 


74 


41 


28 


Items 


64 


29 


24 


Pass/Fail 








Total Test 


79 


58 


58 


Domains 


61 


37 


29 


Objectives 


61 


30 


22 


Past Performance 








Average Scores 


73 


57 


54 


Number or % Passing 


72 


50 


47 


Number or % Failing 


64 


44 


42 



II, Which of the following ways of reporting student strengths and weaknesses 
is most helpful to you? (CIRCLE ONE ONLY) 



12 

15 




T able 6 Continued 

24 A. Rel£,tive to the student's total score (e.g., one domain score 
is lover than you vruld have expected » given the student's 
total score) 

62 B. Relative to the passing score (e.g., one or more domain scores 
are below a certain standard) 

13 C. Relative to other students' scores (e.g., this student scored 
higher than the average student on one domain but lower on 
another) 

III, Which of the following would you be willing to do in order to get more 
information about students' strengths or weaknesses at the objective 
level? (CIRCLE ALL THAT APPLY) 

15 A. Give a longer test 

50 B. Give a test that covers fewer objectives but covers each more 
completely 

71 C. Give follow-up tests for low-scoring students 
13 D. Other 

Teachers were primarily interested in individual students' total scores 
(Part I. A. item 1 - 65%). They were surprisingly less interested in pass/fail 
(38%) objectives (38%) or items (18%). Since objective information is 
traditionally the stuff of which diagnoses are made, let us turn our attention 
to information about strengths and weaknesses. In Part I.B., there appeared to 
be approximately equal interest in all four general areas (total score, 
strengths/weaknesses, pass/fail, and past performance). In short, teachers 
seemed to be moderately interested in everything but not greatly interested in 
any one feature of a potential report. But when forced to choose among these 
options (Part lA) teachers clearly favored generalities over specifics. 

Turning to the questions on page 2 (II and III in Table 6), teachers 
agreed with counselors that strengths and weaknesses should be reported in 
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absolute terms (II* B - 62%). In order to receive more detailed objective 
information, 71% of teachers would give follow-up tests to students who fail 
the functional tests; 50% would give a test that covers objectives with more 
items per objective tested (response B) • Only 15% would give a longer test. 

Parents and sttidents . Parents were most interested in their children's 
test scores (71%), whether they passed or failed (62%), and topics the child 
needed to study (62%) • They were relatively uninterested in which questions 
their children missed (15%), parts of the test on which their children did well 
(26%), and comparisons of their children with other students (37%). The 
picture is fairly clear. Parents wanted to know, in very general terms, ijow 
their children performed. In more specific terms, they wanted to know whether 
their children passed or failed, and if they failed, how to pass the next time. 
There was little interest beyond this point. 

Stuuwflt responses were fairly similar to those of parents. They were 
primarily concerned with topics to study (65%), pass/fail information (63%), 
and total score (62%). They were less interested in domain scores (28%), 
objective scores (31%), and scores compared to those of other students (31%). 
Table 7 summarizes ^he responses of parents and students to all items. 



Table 7 

Questionnaire Responses of Parents and Students 
(Entries are Percentages) 



Parents Students 



Statement 


(N « 99; 




!• 


A statement telling whether you (your child) 








passed or failed 


H 0 
0/ 


0 J 


2. 


Your (child's) total score for a test 


/i 




3. 


Your (child's) scores on the domains tested 


C/i 


9Q 


4. 


Your (child's) scores on the objectives tested 


41 

t X 


31 


5. 


A list of the numbers of the questions you 








(your child) missed 


15 


35 


6. 


Parts of the test on which you (your child) 








did well 


26 


41 


7. 


Parts of the test on which you (your child) 








did poorly 


56 


60 


8. 


Your (child's) score compared to other 








students' scores 


37 


31 


9. 


Your (child's) score compared to the passing 








score 


51 


52 


10. 


Topics you need (your child needs) to study 


62 


65 



Results of Review of Capabilities 

Score reporting has become a high-tech Industry unto Itself. We have 
become so accustomed to computer generated » laser printed, custom designed 
documents that we sometimes fall to consider the possibility that the 
technology Is not universally available. Our review of Maryland's score report 
producing capability had two foci, machines and people. 

At the time of the study, the MSDE had recently purchased a Hewlett 
Packard 2680A laser printer. Linked to a mainframe comp^^ter system (HP 3000 
Model 64) the laser printer was capable of producing totally Individualized 
score reports at the rate of about one per second. The HP laser printer prints 
a page image at a time In exactly the same way that a page of text and graphics 
would appear on the screen of a video display terminal and about as fast. 
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But who operates the machines? The HP laser printer presented a special 
challenge to the Program Assessment Branch of the MSDE because the printer must 
be programmed along with the computer that scores the tests* The language 
traditionally used for producing score report programs was different from that 
used in the new laser printer. Even if the MSDE had made no changes in their 
score reports, they would not have been able to produce them until someone 
bridged the language gap between the scoring programs ^^d the printing 
programs. This was no small undertaking. 

Summary 

One consideration sometimes overlooked in the literature on score 
reporting is the fact that most score report recipients deal with more than one 
testing program. Some tests are diagnostic; some are for accountability, and 
some are for other purposes. Maryland users correctly identified the 
Functional Tests as being strictly associated with accountability. Their 
information needs and interests reflected this understanding. These users are 
probably not atypical of score report users in general. Given this fact, we 
designed forms that emphasized the accountability function of the tests but 
incorporated more than simple pass/fail information. After all, accountability 
is an ongoing responsibility, not just an annual event. 
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Designing the Reports 
Given four levels of reporting and seven content areas* it was necessary 
to design 28 separate reports. For the sake of simplicity and continuity » the 
following presentation focuses on a single content area (reading) across four 
reporting levels (student, class, school, LEA). 

Parents and Students 

As noted previously, parents and students were primarily interested in 
total score, pass/fail information, and topics to study. Figure 1 presents a 
report for a fictitious student Mary L. Student. This report la reduced to 6A% 
of its original size. The actual report is 8^ x 11 inches and the student's 
name is printed in Z4-point bold type (k inch high). The narrative summary 
would have indicated which topics Mary needed the most help in if she had not 
passed. 

A letter to parerts describes the purpose of the tests and provides 
background related to Interests expressed by studc .cs and parents in the 
survey. The description of reading domains is different from most. These 
descriptions are in terms of test questions, rather than instructional 
activities, again, in response to concerns of parents and students. 
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Figure 1 

REPORT FOR: MARY L. STUDENT 

SCHOOL/CODE: ALLECMNYHIOHOeW 



DISTRICT/CODE: 


ALLEGANY COUNTY 01 


GRADE: 


0 


DATE TESTED: 


FALL, 1984 


SUBJECT: READING LEVEL II 


PASSING SCORE IS 340 


YOUR TOTAL SCORE 350 


PASS?YES_ 



SUMMARY 

YOU HAVE PASCED THE MARYLAND FUNCTIONAL READING TEST. 
YOU SHOULD MAINTAIN YCUR SKILLS IN THE FOLLOWING AREA(S): 

FoilowtnQ Directions 

Locatino information 

Malnldta 

UtInoDttailt 

UndtrttandinQ Forms 

SEETHE INFORMATION BELOW FOR A MORE COMPLETE EXPLANATION OF THIS TEST 



TO THE PARENT: 

Your cMkt rtctntly took thf M«ryl«fKt Functional ftoaUing 
THt. Tht rotuHt art shown abovt. Thoit tMiS ara gbttn lo all 
Maryland ttu0fntt to tftttrmint whtthtr or not tt>«y havo 
•cQuirttf cortain ttsic tkm In foodinQ. mathomatics. writing 
•Atf eitiztnahip. 

In W2, ttto Maryland Oonorai Ataambty ptsaod tro *tdu- 
eationai Accountability Act." To carry out thia law. tho Siata 
■oard of Education ^tablishtd frojKt latic. attiing bans 
taquiramrit for high school graduation. At tho aamt tfma tho 
•lata aoard of Education tttabliahod tha Marytand Functional 
Totting Program All Maryland public Mgh school studtntt 
mutt oast thatt ttttt in ordor to gMuata from high Khool. 

Tha ttata ioc^ of Education sti potting tcortt for ttch 
Ittt sfttr hosrhg from :?«c*»^. portnta. and cHKont con* 
comoo about tho tducation of Maryland atudtntt Students 
who do not pott tht totta gra givon titra htip In tchool and 
Ihtn art aMowtd to rttaiit ttia tttta thty faHtd. 

Each Ittt eevtrt ent tub|tct. Each tub|tci la madt up of 
gavtrat domaina. Hating it battd on total tatt acora. Studtnis 
do not patt or fait individual domaint. Whtthtr atudtntt pass 
m fail tht Ittt. thty may ttiti havt atrangths or watkntttts in 
•nt or mart domtlns Thast domaint art dtteribad on thit ptgt 
In itrmt of tht typts of quastions studtnts might bt asiitd 
Additional Information about tht domtlnt. tht pttsing seors. 
and how you etn htip your child do batttr m rttding it avtlltbit 
•I your child's school. 



HEADING DOMA!NS 

I Mrtetltiit: Oivtn dirtct»ons that ara aitt>*r picturts 
or words, tht ttudtnt will Idtnlify tht pioptr course o' 
tction, Outttions mty include road signs rtcipts. msiruc 
tions for optrtting tpplitticts. dirtctions givtn in stvtrti 
tttps.orskniltrittms. 

Laetlinf Mfarmttlon: Givtn a rtftrtnct or rasourct the 
ttudtnt will locttt tptcifitd mformttion Ouettioni mty 
Ctll for mformtlion locattd in ttbfes of conttnti Indexes 
footnotes, bibliogrtphits.cr dctttiogs t.^ simiitr iccttions 

Mtin Wot: Qlvtn a raading ttiection. tht student will identify 
tht main Idtt which mty bt tlther sttted or irr.phed 
Outttions mty Inciudt ptssages from booi^s mtnutts 
Itgtl tfocumtnts. ntwsptptr trticies ^tmphiets or similar 
tourcts 

UtlMg OtttUt: Givtn t rttding stitction. the student will loctte 
and ute deitiis ts directed Outsnons mty tsit the stud^ni 
to list dtltits in thtir ptcper order, to citssify dettits 0( to 
compart dtttils 

Understanding ftrms: Givtn t form or t portion of t foim 
tht ttudtnt Will tail where certain information should go 
Forms may include incontt tax forms. It^surance forms 
tOCitt security forms JobtppiiCtttonforins or Similtf forms 
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School reports were designed for tc^achers and counselors. Because special 
instruction was to be provided for students who scored below a cutoff, the 
reports were designed to group students together. Note in Figure 2 that 
information not directly useful to teachers or ccunaelors in dealing with this 
group is absent. Thus, for example, historical data are absent. A User's 
Guide was designed to describe the tests and their uses in detail. Thus, 
explanatory details iare missing (cf. student /parent report). 




Figure 2 

SCHOOL REPORT FOR: ANY SCHOOL 



DATE TESTED: FALL 1&84 
LEA/OCDE: ALLEGANY COUNTY 01 
AREA: 



SUBJECT: READING 
LEVEL: M 
GRADE: 9 



GROUP ONE: TOTAL SCORE 
340 OR HIGHER 



STUDENT NAME 



MARY DAVE 
GAIL LESH 
HENRY SCHERICH 



GROUP AVERAGE 
NUMBER NEEDING IMPROVEMENT 
PERCENT NI:EDING tbiPROVEMENT 



FOLLOWING 
DIRECTIONS 



LOCATING 
INFORMATION 



MAIN 
IDEA 



USING 
DETAILS 



UNDERSTANDING 
FORMS 



TOTAL 



STUDENT PERCENT SCALE PERCENT SCALH PERCENT SCALE PERCENT SCALE PERCENT SCALE SCALE 
ID NO. CORRECT SCORE CORRECT SCORE CORRECT SCORE CORRECT SCORE CORRECT SCORE SCORE PASS** 



COOOOl 

000002 
000003 



65 341 
75 352 
84 368 


68 335* 
76 344 
83 354 


59 361 
68 370 
78 381 


66 350 
73 358 
80 368 


73 360 
80 371 
88 387 


350 YES 
360 YES 
372 YES 


'7r^351 ^ 
0 
0 


"75 342 
2 

10 


69 372 
1 
5 


75 363 
0 
0 


80 371 
1 
5 


359 



GROUP TWO: TOTAL SCORE 
BELOW 340 



FOLLOWING 
DIRECTIONS 



LOCATING 
INFORMATION 



MAIN 
IDEA 



USING 
DETAILS 



STUDENT NA)#E 



STUDENT 
ID NO. 



UNDERSTANDING 
FORMS 



TOTAL 



PERCENT SCALE PERCENT SCALE PERCENT SCALE PERCENT SCALE PERCENT SCALE SCALE 
CORRECT SCORE CORRECT SCORE CORRECT SCORE CORRECT SCORE CORRECT SCORE SCORE PASS? 




^INDICATES NEED FOR IMPROVEMENT 



Note that even among the group scoring above the cutoff, some students 
will need review or remediation. The under Locating Information for Mary 
L* Student Indicates such a need. On the Group Two Report, it becomes 
immediately obvious that Main Idea caused problems for most of the group. This 
section thus highlights group as well as individua'". needs. 

Figure 3 Illustrates the last page of a school report. This page 
summarizes all results and indicates general strengths and weakness. It may be 
used for classroom comparisons (e.g., my class vs. the rest of the school). 
Note that there is no class by class breakdown. 



Figure 3 

SCHOOL REPORT FOR: ANY SCHOOL 

DATE TESTED: FALL 1984 
LEA/CODE: ALLEGANY COUNTY 01 
AREA: 




SUBJECT: READING 
LEVEL: II 
GRADE: 9 



FOLLOWINO LOCATINO MAIN USING UNOERSTANO!NQ 

DIRECTIONS INFORMATION IDEA DETAILS FORMS 





PERCENT SCALE 
CORRECT SCORE 


PERCENT SCALE 
CORRECT SCORE 


PERCENT SCALE 
CORRECT SCORE 


PFRCENT SCALE 
CORRECT SCORE 


PERCENT 
CORRECT 


SCALE 
SCORE 


SCHOOL AVERAGE: AU STUDENTS 


72 349 


69 339* 


86 395 


59 342 


87 


385 


NUMBER NEED4N0 IMPROVEMENT 




10 


15 


\l 


8 


PERCENT NEED4N0 »IPROVEME^fT 


15 




38 


28 


20 



NUMBER OF S TUDENTS TESTED: 
NUMBER OF STUDENTS PASSING: 
PERCENT OF STUDENTS PASSING: 



40 
20 
50 



School Stnnmary Report 

Building level administrators wanted LEA compai.ison as well as historical 
data. The main feature of Figure 4 is that it contains very few numbers. 
Nothing appears that was not requested by most principals. The result is that 
principals can Immediately check this year's results against last year's and 
against the other major benchmark of success, the competition. Yet in this 
report, there is no school-by-schocl breakdown. We discovered a balance 
between no comparative data at all and the kinds of invidious comparisons one 
customarily finds in the local newspapers. 




Figure 4 

SCHOOL SUMMARY REPORT FOR: ANY SCHOOL 



PRINC|P*»,COPY 



DATE TESTED; FALL 1984 



SUBJECT: READING 



LEVEL n GRADE: 9 



SCHOOL • LEA COMPARISON 



FOLLOWING LOCATIMO MAIN USINO 
IMKECTIONS INFOHilATION lOCA li..AIL8 
STUDENTS p;£RCENT MEAN PERCENT MEAK PERCENT MEAN ^RCENT MEAN 
Tperrp Aflf>VFl^ A^wuii RnfWlF ABOVE 340 SCORE ABOVE340 SCORE 


UNDERSTANDINQ 

PERCENT MEAN 
ABOVE 340 SCORE 


TOTAL 
PERCENT MEAN 
PASSING SCORE 


SCHOOL 


40 


55 


349 


50 


339 


70 


395 


50 


342 


63 


385 


50 341 


LEA 


240 


50 


340 


70 


379 


60 


36^^ 


50 


340 


60 


372 


50 340 










SCHOOL PERFORMANCE BY YEAR* 












STUDENTS 
TESTED 


F0U0W1N0 LOCATINO 
CNRFCTIOHS iNFORMATtOH 
PERCENT MEAN PERCENT MEAN 
ABOVE340 SCORE AeOVE340 SCORE 


MAIN USING 
lOCA DCTAILS 
PERCENT MEAN PERCENT MEAN 
ABOVE 340 SCORE ABOVE 340 SCORE 


VMOCRSTANOINO 

FORMS 
PERCENT MEAN 
ABOVE 340 SCORE 


TOTAL 
PERCENT MEAN 
PASSING- SCORE 


1964 


•40 


55 


349 


50 


339 


70 


395 


50 


342 


63 


385 


50 341 


1963 


38 


50 


341 


50 


338 


60 


365 


50 


340 


60 


361 


46 336 


1962 


42 


55 


350 


. 50 


337 


50 


345 


43 


328 


54 


346 


42 327 



•INITIAL FAU DATA ONLY 



Principals were content to know where their schools stood relative to 
other schools in generals If they were below average, it helped to know they 
had Improved over last year. If they were below average and posted a decline 
from last yearns results* they knew they voulf have some explaining to do. The 
accountability function was served in a way that all parties understood and 
accepted » even when the results were unpleasant, 

LEA Reports 

Two LEA reports were designed » reflecting the different information needs 
of program managers (local accountability coordinators) and general 
administrators and elected officials (superintendents > assistant 
superintendents, school board). The LEA Report is a school by school summary 
for the local accountability coordinator. The LEA Summary Report parallels the 
School Summary report by providing LEA/state comparisons and historical data. 

Figure 3 dws the LEA Report. Here we see the percent passing and mean 
score for each school in the district. The date and name and level of the test 
are shown as well as numbers of students tested. The local accountability 
coordinator (LAC> is responsible tor assuring that each school performs up to 
standard. This person also needs to know where major weaVtiesses lie, either in 
specific domains across schools or in specific schools across domains. This 
report satisfies those needs ^ 
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LEA REPORT FOR: ALLEGANY COUNTY 



DATE TESTED: FALL 1984 SUBJECT: READING LEVEL: II GRADE: 9 



SCHOOL NAME 


CODE 


STMDENTS 
TESTED 




FOLLOWiNQ 
DIRECTIONS 


LOCATING 
INFORMATION 


MAIN 
IDEA 


USING 
DETAILS 


UNDERSTANDING 
FORMS 


TOTAL 


ANY SCHOOL 


0613 


40 


Vo PASSING 

MEAN SCALE SCORE 


55 
349 


50 
339 


70 

395 


50 
342 


63 
375 


50 
341 


MY SCHOOL 


0614 


61 


% PASSING 

MEAN SCALE SCORE 


48 
331 


80 
396 


50 
340 


51 
340 


55 
359 


51 
340 


YOUR SCHOOL 


0615 


96 


% PASSING 

MEAN SCALE SCORE 


51 
342 


67 
361 


63 
368 


48 
339 


61 

3/0 


51 
342 



































LEA TOTAL % PASSING 

MEAN SCALE SCORE 


50 
340 


70 
379 


60 
365 


50 
340 


60 
372 


50 
340 



NUMBER OF STUDENTS TESTED: 240 
NUMBER OF STUDENTS PASSING: 120 
PERCENT OF STUDENTS PASSING: 50 



In figure 6 we see a sample LEA Summary Report. Again we see comparisons 
to the larger system (state) and with previous years. There is very little Ij 
detail here; for example* the school by school comparisons are missing. Yet 
the superintendent (who receives this report) does have access to another H 
report which provides these comparisons if they are needed. Further, the 
superintendent is given concrete evidence for presentations to the school board 



and ultimately ^ o local media, and In very concise fishlon. An all Important 
context Is provided for Interpretation and discussion. The LEA State 
comparison Invites discussions of local vs. state average per-pupll 
expenditures and the like. The LEA State comparison Invites discussions of 
local vs. state average per-pupll experdltures and the like. The-LEA 
performance by year ^ rovldes a framework for discussing changes In policies and 
programs over the past three years. In both cases » the discussion Is clearly 
framed and may proceed In a productive manner. Contrast this situation with 
the unadorned ^•Half of County Flunks Test** headline seen some years ago in a 

Figure 6 superintencentcopy 
LEA SUMMARY REPORT FOR: ALLEGANY COUNTY 

DATE TESTED: FALL 1984 SUBJECT: READING LEVEL: !l GRADE: 9 




LEA-STATE COMPARISON 



TOTAL 



FOLLOWINO LOCATIMO MAIN OSINO 

DIRECTIONS INFORMATION IDEA DETAILS 

STUDENTS PERCENT MEAN PERCENT MEAN PERCENT MEAN PERCENT MEAN PERCENT MEAN PERCENT MEAN 



UNDERSTANDINQ 

FORfJIS 



LEA 


240 


50 340 


70 379 


60 365 


50 340 


60 


372 


50 


340 


STATE 


65»385 


51 340 


56 351 


55 349 


57 359 


54 


349 


53 


347 



.EA PERFORMANCE BY YEAR* 



FOLLOWINO LOCATINO MAI>i I^NO ONDERSTANDINO 

DlRECnOHS INFORMATIOH IDEA DETAILS FORMS TOTAL 

STUDENTS PERCENT MEAN PERCEM .^EAN PERCENT MEAN PERCENT MEAN PERCENT MEAN PERCENT MEAN 

TESTED A^eSosZ^^^ A^VE Sio ^RE ABOVE340 SCORE ABOVE 340 SCORE ABOVE 340 SCORE PASSING SCO?^E 



1964 
1983 
1982 



240 



245 



247 



50 340 



50 339 



41 328 



70 379 



60 368 



55 350 



60 365 



55 350 



4b 33/ 



50 340 



50 341 



50 340 



60 372 



56 351 



52 345 



50 340 



50 340 



46' 336 



•INITIAL FALL DATA ONLY 



Conclusions 

None of these reports contains a great deal of information. No 
information below the domain level is given in any of f' em. The student report 
contains a description of the domains from an item perspective. Higl ?r levels 
of reporting are backed up by a series of documents including the Declared 
Competencies Index (DCI) and a User's Guide. The DCI describes each domain and 
objective in detail, while the User's Guid<» offers aid in interpretation and 
use of results. 

Each report is oriented to a specific audience. Each report therefore 
contains only that information the recipient han shown a need to have. While 
not every item on every report was specifically requested by iuj users, each 
enhances the usefulness of the report for some important purpose described by 
u^sers. Thus, for example, while parents did not specifically ask to know the 
name and level of the test, such information is absolutely crucial in a state 
where parents may recieve two or three sets of results in a given school year. 
More importantly, the specific items requested by parents, teachers, and others 
are there, an acknowledgement of the rightful ownership of the testing program. 

The score reports presented in Figures 1-6 would not score very many 
points on the checklist devised by Mills and Hambleton (1984) . So how do these 
reports differ from the dozens of others we reviewed? 

First, visual clutter is reduced to an absolute minimum. Particularly for 
students and parents, the most important items of information are in large bold 
type and only two numbers (Passing Score and Your Total Score) ever appoar (the 
Writing Level II report contains four numbers). 

Second, diagnostic information is given all the attention it deserves, and 
no more. There is just so much Information one can squeeze out of fifty or 
sixty test items. Uaers and lesijmers agree that this test is primarily an 
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accountability test» not a diagnostic test. The limited amount of diagnostic 
information available is repovtad only where it is likely to be helpful: on 
the reports to teachers. These same teachers are given excellent support 
materials to perform .neir own more detailed diagnoses if they so desire. 

This gives rise to the third point. A system has been designed around 
these reports in such a way that all pieces interlock and support one another. 
The User^s Guide provides thorough background on score interpretation at 
multiple levels. The Declared Competencies Index defines each objective in 
great detail. A series of cross-referenced manuals » guides, and handbooks 
giv»s samples of instructional as well as diagnostic activities. The larger 
structure of Project Basic » whose objectives are assessed by the functional 
tests > provides for interpreting results and for setting new goals within a 
framework familiar to anyone associated with a Maryland school. 

This score reporting systnm recognizes the responsibilities and 
information needs of all its audiences. Consider the following statements 
taken directly from the student and school reports: 

You need to improve your skills in using details. 
* Indicates need for improvement 
From student to superintendent » all have some degree of responsibility for 
improving basic skills. Those responsibilities are acknowledged and relevant 
information is articulated in a way that helps each meet his or her own 
responsibilities • 
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